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Abstract 

This project devebped a con^ptualization of higher order thinking, dimensions of 
classroom thoughtfiUness, and an assessn^nt exeicse applicable to diverse topics taught in 
more than 70 classes in 11 high schools. Generic qualities of classroom thoughtfiilness were 
not generally associated with the persuasiveness of stiKlent writing on a constitutional issue. 
But the impact of classroom thoughtfulness on hi^^ order thinking in subjects that teachers 
actually taught was not assessed. The study illustrates issues in devising ot^rvation schemes 
and assessment tasks that at once honor diverse forms of domain-specific competence in social 
studies, but that also yield common indicators of teaching quality and student performance. 
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Secondaiy school social studies varies considerably between schools, between courses 
within schools, and between sections of the same couise within a school The diversity s 
nourishcxi by the pluralistic content of the currkmlum - which includes di«:iplines of history, 
gTOgraphy, political science, economics, sociokigy, psychology, anthropology, many specialties 
vtrithin these disciplines and a host of areas beyond formal acactemic disciplines considered 
legitimate areas of study (e.g., ethnic and women's studies, ^bal education, environmental 
studies, multicultural studies, law-related education). Teaclrcrs* efforts to shape content to 
the diverse abilities and inter^ts of students furtl^r dififerentiates instruction. I^ally, the 
varied political interests of teachers and local communities make the setection of content even 
more controversial and non-uniform. Yet, in spite of the diversity, aiucatcrs often agree that, 
regardless of the content studied, a major purpc»e of social studies instruction should be to 
help students think critically and creatively about the subject and that careful, disciplined 
thinking should eventually be applied to pubUc issues that confiront citizens in a democracy. 

In the midst of disparity among specific instructional goals, how might we assess 
progress on the more general goal of promoting students' thinking? Some common indicators 
are needed to respond to the growing demand for school, district, state and national 
accountability. Common indicators could reveal not only the general qualities of student 
thinking, but also provide information to identify inequities in instruction and opportunity to 
learn. Common indicators for both instructional quality and student thinking might help to 
unify the profession around some goals, while, at the same, preserving diversity in the 
teaching of more specific content Diversity in specific content between classes, schools, 
districts and states is consistently ad^tjcated tu respond to students' personal backgrounds, to 
support cultural pluralism, to foster teacher conunitment and creativity, and to adequately 
represent the many discipline of the social studks field. 
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In the study reported here, we addressed this problem by identifying "generic" qualities 
of classroom thoughtiiilness that could be eiqiected to promote students* thinicing across a 
wide range of social studies courses. We also developed a test for assessing the quality of 
student thought on publk: issues which did not depend upon students* prior nusteiy of 
specific content Elsewhere we have reported more extensively our conception of thinking 
and the rationale for the assessment of classroom thoughtfuhiess (Newmann, 1990a), the 
devebpment and scoring of the test (Newmann, 1990c), and several other aspects of the 
study.^ Following a revisw of tim material, the focus here will be on the relationship of 
classroom thoughtfiilness to students' higher order thinking as assessed by the test 
I. What Is Higher Order TUnidiig? 

Researchers and educators ha^ advocated many co«:eptions of thinking: critical 
thinking, divergent or creative thinking, reasoning (moral, practical, deductive, inductive), 
probtem-sohang, decision-making. These can all be subsumed under a more general 
distinction between higher order and lower order thinking. Higher order thinking is defined 
broadly as challenge and ^anded use of the mind; lower order thinking represents routine, 
mechanistic application, and limited use of the mind. Challenge or expanded use of mind 
occurs when a person must interpret analyze, or manijmlate Information, because a question 
to be answered or a probtem to be soh^ed cannot be resolved through the routine application 
of previously learned knowledge. In contrast, "lower order" thinking generally invoh^es 
repetitive routines such as listing information previously memorized, inserting numbers into 
prevk}usly learned formulae, or applying the rules for footnote format in a reses^.^h paper. 

^Topics covered include te«;hers' thinking (Onosko, 1989, 1990); principal and 
departmental leadership (McCarthy & Schrag, in press; Kmg, 1991); student engagement and 
cognitive challenge (Stevenson, in press; Newmaim, in press b); organizational features 
(Ladwig, 1991); empirical profiles of classroom thoughtfulness (Newmann, 199(»>); and 
general summaries of the proja:t (Newmatm, in press a and b). 



Chdlenging problems can appear in many fonns in all curriculum subjects. They may 
le«l to single, correct, aixi well-deGned answers or to multiple, ambiguous, conflicting 
solutions. The challenges may involve different IdiKb of inquiry (logical, empirical, aesthetic, 
ethical), difTercnt forms of expression (oral, written, non-verbal), different types of 
intelligence (verbal, mathematical, kinesthetic, interpersonal). 

No particular question or problem, however, necessarily teads to higher order thinking 
for all stiuients. For one person, trying to understand aad foUcnv a bus schedule may require 
higher order thought, but for another, the same task may be routii^ In this sense, higher 
order thinking is relative: to determine the extent to which an individual is invoked in higher 
order thinking, one would presumably need to know much about the person's history. 
Furthermore, to assess the extent to which an imiividual is participating in the analysis, 
interpretation, ain! manipulation of information, one would want to "get inside" the person's 
head or experience hi^er subjective state of thought 

This definition poses an operatwnal prc^lem. It is difficult to determine the extent 
to which a person is involved in higher order thinking, and difficult also to judge the quality 
of that thinking. Teachers who interact with several students at once have little opportunity 
to diagnose students' individual mental states. Instead, they must make assumptions about 
the ppor knowledge of groups of students and about the kinds of mental work that particular 
tasks are likely to stimulate. The teaching of thinking, therefore, is a rather imprecise 
enterprise. The best we can do is to engage students in what we anticipate wiU be challenging 
problems, to guide their manipulation of information to soh^ them, and to support their 
efforts. 

But this conception of higher order thinking has several positive features. 
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(1) Any person, young or old, regardless of experience, can participate in higher 
order thought Students will differ in the kinds of chaltenges th^- are able to master, but all 
are capabk: of confronting a challenge in the interpretatbn, analysis, and manipulation of 
their knowledge. 

(2) It etK»mpasses problem-sohing in a wide range of school subjects, as well as in 
m^n-academic areas. 

(3) Using this conception does not require M%eptance of any particular theoiy oC 
cognitive processing or a particular pedagogy. This is an advantage, because solid knowledge 
on the best techniques for the promotion of thinking does not exist The effectiveness of 
technique wiU probably depend on the nature of the mental challenges presented and 
characteristics )f the students exposed to thent Furthermore, this conception is hospitable 
to providing students with thr^ resources recognized widely in the literature as important: 
content knowledge, intellectual skiUs, and dispositk^ns of thoughtfiilness. 

Merely presenting students with higher order challenges will not necessarily help them 
devekjp the competence to meet the challenges successfully. Research on the nature of 
thinking (e.g. as summarized by Walsh and Paul, 1987) indicate that for students to cope 
successfully with higher order chaUenges, they need a combination of in-depth knowledge, 
intellectual skills, and attitudes or dispositions of thoughtfiilness. Building upon my previous 
review of literature (Newmann, 1990a) I summarize here key arguments that can be made for 
each of these critical resources. 

The Knowledge Argument. Consider a teacher trying to help students answer the 
question, "Were the American colonists justified in using violence to secure their 
independence from England?" Regardless of what side the student takes, a successful 
respome depends on in-depth knowtedge and a>nceptual understanding of the circumstanoK 
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of colonial life under British rule, colomal grtevances and British responses, prim:ipled 
arguments dealing with inalienable rights, taxation without representation, and ethical 
reasoning related to the destruction of property and the taking of human life. Beyond 
substantive knowledge about the historical period, students will need analytic knowledge, for 
example about elements of a well-reasoned argument, distinctions between empirical and 
normative issues, and criteria for judging the reliability of evidei^s. Metai;ognitive knowledge 
may also be important, such as having a systematic approach for organizing one's thinking or 
an awareness of how one's thought processes and p»c«ptions of others - in the heat of 
discussion > might lead to error. Effective applicatk)ns of these forms of kiKTwledge are 
sometimes labeled skills or dispositwns, but since these all can be represented as cognitive 
beUe&, they suggest that knowkxlge itself is the nKsst critical foundation of understanding.^ 
The Skills Argument Knowledge is undoubtedly important, but for the purposes of 
the teaching of thinking, skills are critical because they are the tools that permit knowledge 
to be applied to the solution of new problems. Some skilb vosy be specify to the domain 
under study, and others more generk. To address the problem above intelligently, for 
example, one must be able to detect bias in the documents of cotonial history and logical 
fallacies in inferences and arguments over the justi£k»tion of the American revolutbn. One 
must be able to distinguish relevant from irrelevant information, to anticipate and to respond 
to arguments in opposition to one's own, to state one's views clearly and persuasively. Skills 
therosehres may be construed or labeled in a variety of ways, but the main point is to 
recognize their role as cognitive processes through which knowledge is put to work. In 
practice, knowledge is usually only transmitted from teacher to student without expecting the 



Various arguments for the centrality of knowledge to reasoning have been made by 
Glaser (1984), McPeck (1981), and Nickerson (1988). 
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student to manipulate the knowledge to solve higher order challenges. Untess the processes 
of using icnowledge, le., skills, are stressed, higher order thinking is likely to be neglected and 
the knowledge transmitted to remain inert Perhaps for this reason many educational 
reformers prefer not to advocate the teaching of thinking, but instead the teaching of thinking 
skills. ^ 

The Dapositions Arpiment Without dtsposittons of thoughtfulness, neither 
knowledge nor the tools for applying it are likely to be used intelligently. If raising questions 
about the justification of the war for American imiependence threatens patriotic feelings, this 
c»uld jeopardize dispassionate inquiry. Some peopte may avoki almcst any argument to 
protect themselves frcm uncomfortable feeUngs of conflict Those who emphasize the 
importance of dispositions suggest several crucial traits: a persistent desire that claims be 
supported by reasons (and that the reasons themseh^ be scnitini^}; a tendency to be 
reflecthre - to take time to think problems through for one%lf, rather than acting impulsively 
or automaticaiiy accepting the views of others; a curiosity to expk>re new qt^tions; and the 
flexibility to entertain alternative and original solutions to problems. Thoughtfulness thereby 
involves attitudes, personality or character traits, and general values and beliefs or 
epistemotogies about the nature of knowledge (e.g., that rationality is desirable; that 
knowledge itself is socially constiuctedL subject to revision, and often indeterminate; and that 
thinking can lead to the understanding and solution of problems). Without dispositions of 
thoughtfulness, knowledge and skills are likely to be taught and applied mechanisticalty and 
nonsensically. Of the three main resources, dispositions have attract^ the least attention in 
professional literature, but a good argument can be made that dispositions are central They 



Various arguments for skiUs as the most central resource in thinking have been made by 
Beyer (1987), deBono (1983), Hermstein et aL (1986), Marzano et al. (1988). 
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seem to establish both the ^ to think and to cultivate ineffabte qualities of judgment that 
steer knowledge and skills in productive directk>ns.^ 

It is important that teachers design instruction explicitly to help students acquire and 
to use in-<iepth knov^ge, skills ai»l dispositk>ns of thougbtfulncss to solve higlM^r order 
challenges. It is not possible to estab&h a defensible hierarchy among the three resources, 
but all three are needed. The observation scheme to be presented next is an attempt to 
capture teachers' efforts to devetop knm^^f^ skills and disp(»ttlk>ns, without giving center 
stage to any oik resource, and abo to refrain from presoibing the precise kinds of 
knowledge, skills and dispositions that shouki be promoted for the teaching of each subject.' 
The reasoning behiivl this choke is e9q}lained in the next section that presents the framework 
for assessing classroom thoughtfulness. 
n* Dcvdofrf^ ladkators oi ClassroiMn Tto^tfalacss 

What kinds of indicators wouki provkte informatkin on the extent to which higher 
order thinking was promoted in classes stiKiying a variety of social studies subjects. Because 
it was logistically impc^sible to examine the actual thinking of individual students during the 
lessons, a more general tool was needed for descnbing higher order thinking in the lesson as 
a whole. But how sped" : shouki the criteria be? 

Interviews with histoiy and social studies teachers indicated that highly spmfic Usts 
of knowksdge, skills and dispositbm would be unlikely to facilitate widespread consensus. 
Instead, socidl studies teachers are likely to support a plurality of types of thinking, but even 



Various arguments for dispositions as a (antral resource in thinking have been made by 
Combleth (1985), Dewey (1933), and Schrag (1988). 

^ose who emphasize interaction ami interdependeiKx; among th^ resources include 
Bransford el al (1986), Ennis (1987), Grceno (1989), Perkins «nd Satomon (1989), and Walsh 
and Paul (1987). 
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these will be grounded primarily in the teaching of different subjects. Thus, a broad 
conception of thinking, adapuble to a variety of content and skill objectives, is more likely 
to interest a diverse population of high school teachers. 

Rather than translating thinking into sped&c knowledge problems, skills and attitudes 
for students, the project staff began by asking what observable qualities of classroom activity 
would be most likely to help students achieve depth of understanding, intellectual skflls, and 
dispositkms of thoughtfulness. Thus, we moved &om a consideration of the nature of 
thinking in individual stucknts to the pronrating thoughtfulness in classrooms. Thoughtfuhiess 
includes both presenting students with higher order challenges and helping them apply 
knowledge, skills and dispositions to soWe them. Emphasizing general qualities of classroom 
talk and acthaty rather than highly differentiated behavk>rs helps to avoid fragmentation in 
teaching which itself can undermine student thinking. A more general approach may also 
hold more promise both for students to soKe new problems and for teachers to promote 
thinking across diverse lessons. 

A broad set of criteria can strike at the heart of an underlying malady identiGed in 
many studies. At best, much classroom activity fails to challenge students to use their mii:ds 
in §222 valuable ways; at worst, much classroom activity is nonsensical or mindless. The more 
serious problem, therefore, is not the failure to teach some specific aspect of thinking, but 
the profound absence of thoughtfulness in classrooms. Even programs designed to teach 
thinking skiUs can fail to promote thoughtfuhiess. Our general conception of thinking can 
be used to address this basic issue. Ultimately, of course, teachers must focus on the content- 
specific activities that enhance understanding of their subjects, but the point here is to arrive 
at a general framework through which classroom behavior can be interpreted as promoting 
or undermining higher order thinking. 
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In devising indicators of dasuDom tboughtfiilness responsive to the points above, we 
initially rated lessons on 15 possibte dimensions of classroom tlujughtfulness summarized in 
Table 1. Each was used to make an ovc»b!1 rating of an c^iserved lesson on a 5-paint scale 
from 1 - "a very in«xurate" to 5 = *a very acctirate" description of this lesson. After 
observing these qualities in 160 lessons in five "select" social studies departments and further 
examining them from a theoretical point of view, we chose the six main dimensions described 
below as most fundamental.^ 

/. There was sustained examinaam of a few tofria rather than superficial 
coverage of many. 

Mastery of higher order challenges r^uires in-depth study and sustained 
conoentraUon on a limited number of topics or questbns. Lessons that cover a large number 
of topics give students only a vagt» familiarity or awareness and, thereby, reduce the 
possibilities for building the complex knowledge, skills, and dispositions required to 
understand a topic 

2. The lessm displayed substar%iive coherence and c<mtinuity. 

Intelligent progress on higher order challenges demands systematic inquiry building 
on relevant and accurate substantive knowledge in the Held and working toward the bgical 
development and integration of ideas. In contrast, lessons that te^h material as unrelated 
fragments of knowledge, without pulling them together, undermine such inquiry, 

3. Students were given an appropiate amount of time to think, that is, to pre[mre 
responses to questions. 



^Thc devebpment of these indicators and setecUon of the six most critical are described 
more fuUy in Newmann (1990a; 1990b). Sec also Schrag (1989). 



Thinking takes time, but often recitation, discussion, and written iKsignments pressure 
students to make responses before they have had enough time to reflect. Promoting 
thoughtfiilness, therefore, requires periods of siiemx during which students can ponder the 
validity of alternative responses, devetop more elaborate reasoning, and experience patient 
reflection. 

4. The teacher asked chaOen^a^ questions andfa- structured diaUenga^ tasks 
(g^en the ability level and pr^amticm of the students). 

By our definition higher onler thinking occurs only when stwtents are faced with 
questions or tasks that demand anafysis, interpretation, or manipulation of information; that 
is, non-routine mental work. In short, students must be f»xd with the challenge of how to 
use prior knowledge to gain new knowledge, rather than the task of merely retrieving prior 
knowledge. 

5. The teacher was a model of thoughtfidness. 

To help students succeed with higher order challenges, teachers themseKes must 
model thoughtful dispositions as they teach. Of course, a thoughtful teacher would 
demonstrate many of the behaviors described above, but this dimension is intended to capture 
a cluster of dispositions likely to be found in any thoughtful person. Key indicators include 
showing interest in students' ideas and in alternative approaches to prc^lems; showing how 
he/she thought through a problem (rather than only the final answer); and acknowledging the 
difficulty of gaining a definitive understanding of problematic topics. 

d Students offered explanations and reasms for their conclusions. 

The answers or solutions to higher order challenges are rarely self-evident Their 
validity often rests on the quality of explanation or reasons given to support them. 
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Therefore, beyond oSiering answers, students must abo be helped to produce explanations 
and reasons to support their oonclusk)ns. 

The six din»nsk>ns were combined into a single scale indicator of classroom 
thoughtfiilness for an observed lesson.^ To estimate inter-rater reliability, 87 lessons in 16 
high schools were observed independently 1^ different pairs of raters drawn from a team of 
six researchers. Considering the six dimensions in the classroom thoughtfiihiess scale, each 
scored from 1 to 5, the overall corrclatk>n between two observers was .76. The raters agreed 
on 64% of the rating and they differed by one point or less on 96% of the ratings, 
m. Testiag Thinking 

What kind of test or exercise would give a meaningful indicator of the quality of 
students' thinking m social studies? Ideally, one woukl want assessment tasks that posed 
novel challenges in understanding the specific subjects studio in each of the c^jserved classes. 
Because of the diversity of topics and teachers' goals, these tests would vary considerably in 
the nature of in-depth knowledge required to soKe the problem, the kinds of skills needed, 
and possibly also the underlymg dispositions. To compare the performances of students in 
different classes, it would be necessary to ^velop common standards on which to evaluate 
the diverse tests. Unfortunately, our project lacked the resources to undertake this 
complicated approach to assessment 

Another alternative was to administer a previously ckvek>ped test of critical thinking, 
such as those reviewed by Arter and Sahnon (19S7). We rejected this on the grounds that 
such tests, which focus primarily on students' logical-<teducttve skills and which are usually 
answered in multiple-choice form, do not reveal the depth of students' understanding of any 

^Jtems on the scak ^ave a reasonably high level of internal consistency (Cronb«:h alpha 
= .82). Exploratory factor analysis and LISREL modeling also grouped these dimensions into 
a distinct construct of thoughtfubess (Newmann, 1990b). 
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particular topic and they do not show how students organize their thoughts in their own 
language.^ 

Our research indicates that when social studies teachers want to promote thinking, 
their aim is not usually to teach and test for discrete thinking sldlU, such as hypothesis testing 
or evaluating the reliability of sources. Instead, they design their daily efforts to fulfill a more 
general visk>n ~ leading students to interpret, analyze, and use their knowledge of histoiy, 
government, geography and the social sciences to undestand the contemporary world. 

The Posttest: Ppmiasiw. W riting on a Con stitutwnal Issue. We devised an exercise 
consistent with this general purpose, not aimed at tl^ specific content taught in any class. 
Students were given a two-page document (see Appendix) that ctescribed a hypothetical court 
case, based on an actual case, invohdng the search of Karen Doctor's purse and locker by the 
high school assistant principal who suspected Karen first of smoking in violation of a school 
rule and then of selling marijuana. FoUowing the case description, background information 
was given on the main principles that courts have used in making decisk>ns about the 
constitutionality of student searches. Students were asked to decide whether Karen's 
constitutional rights were violated and to write a persuasive essay which explained and 
defended their views i^ing information in the reeling. 

Completing the task requires higher onkr thinking, becai^ to succe^, students must 
organize and interpret information in a new way (assuming they have not previously studied 
this issue). Virtually all of the substantive knowledge needed is available within the 
document This is not a test of what students have remembered from social studies, but a 
test of their competence in thinking about social studies content. As explain^ below, an 



*See Norris (1989) for further discussion of this point. 
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analogous task, the case of Anthony aiui Stanton invoking school censorship of a student 
newspaper, was used as a pretest in one of the data sets. 

There are, of c»uxse, a variety of impcntant ways to think ahout social studies content 
that this test does not attempt to assess - for exampte, explaining historkal causation, 
analyzing cultural differences, interpreting economic data, sohdng moral dilemmas, or 
critiquing the use of political power. Social studies assessment slrauld give serious attention 
to all of these forms of thinking. Our test is limited to the kind of thinking invohvd in 
writing a persuasive position on a constitutional kswe. Ths stould not, however, be seen as 
an esoteric or highly specialized fsaxi of social studies. To tbs contrary, it ym chosen 
because it represents a central civic competence and an important objective of social studies. 

According to the research design described later, the test was administered toward ita 
end of the spring semester, and students had about 50 mini cf.^ to comntete it Almost all 
students finished with time to spare. 

Scoring . Students' essays were scored from 1 to 5, based on the foUowing criteria 
which we adapted from the NAEP assessment of persuasive writing (Applebee, Langer, 
Mullis, & Jenkins, 1990). Essays received one of five scores: (1) unsatisfactoiy, (2) minimal, 
(3) adequate, (4) elaborated, or (5) exemplaiy. The ovcrarohine consideration was the degree 
to which a student's response was capable of persuading a reader . Three elements focused 
the assessment: whether or not the stiKlent had a) taken an informed stand, b) provided 
persuasive reasons, and c) elaborated upon those reasons. Sp«nfic points were not subtracted 
for unpersuasive or irrelevant reasons but these could diminish persuasiveness. Presentation 
of faulty assumptions or reasons that undennint: the argument coukl also diminish overall 
persuaswcness. Finally, responses were to be written in sentences; that is, incomplete 
sentences or fragmented lists were considered less persuasive. Fuller descriptions for each 
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of the scores are given below. Example of responses for e«;h are given in Newmann (1990c) 
which gives mote detail on development and scoring of the test. 

1. Unsatisfactory: The stiulent has failed to take a stand on the issue under examination, 
or has taken a stand but has failed to provkte a singte persuasive reason. Lacking a 
persuasive reason, unsatisfactoiy respcmses will necessarily lack elaboration. Overall, 
the response has no chance of persuading the reader. 

2. Minimal : The student has taken a stand on the issue under exfunination and has 
provided at least one persuasive reason, or at least two supportive reasons. Faulty 
aaumptioi^ undermining, or irrelevant reasons could result in an unsatttCaciory score 
if they reduce the persuasiveness of the argument CXerall, the response is unlikely 
to persuade the reader. 

3. Adequate - The student has taken a staiKi and has provkled two or more persuasive 
reasons. Elaboration of reasons is iK^t necessary here. The presentation of only one 
persuasive reason can r^ult in a score "attequate" if useful elaboration is included. 
Undermining reasons, faulty assumptions, or irretevant reasons can possibly reduce the 
score to "minimal". Overall, the response has a chance of persuading the re^r. 

4. Elaborated : The student has taken a stand, has provided two or more persuasive 
reasons, and has provided elaboratbn on at least one of those reasons. Presentation 
of many persuasive reasons (at least 3) without elaboration can also produce this 
score. Undermining reasons, faulty s^umptions, or irrelevant reasons can possibly 
reduce the score. Overall, the respome is likely to persuade the re4..er. 

5. Exemplary : The student*s response meets criteria for (4) above, and demonstrates (a) 
at least two elaborated persuasive reasons, and (b) an argument so clear and coherent 
(i.e., no significant undermining reasons, faulty assumptions or irrelevant reasons) and 
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grammat* lUy correct as to merit publk displff^ as an outstanding accompUshment for 

a high school stiMtent Overall, the response is more likely to persuade the reader. 

To apply the criteria, several additk)nal scoring convention were cteveloped to clarify 
what counts as a persuasive retoon and what substanth« principles in the case are considered 
retevant to the argument for each side. 

To determine inter-rater agreement, different pairs of two raters read 492 tests (375 
posttests, 117 pretests) which amounted to about 29% of the tests completed. The overaU 
correlation between ratings was .80. Raters adiKved exact agreement in 65% of the cases 
aad agreed exactly or missed by only one point in 98% of the cases.' 
IV. Design 

This stucfy was part of a lar^r project to investipte how high sctMX>l social studies 
departments might pronKite higher onler thinking more successfully (see Newmann, in press 
a and b). The strategy was to study social studies departments that varied in department-wide 
efforts to emphasize higher order thinking, and then, by ramparing these, to draw inferences 
about barriers and opportunities for success. Between the fall of 1986 and the spring of 1990, 
the project conducted ahnost 500 observations of lessons, and in-depth interviews with 56 
teachers, the social studies department chairs and the principals in 16 ctemc^phically diverse 
high schools. 

Through national searct^ which invoh^d nominations, phone interviews, and site 
visits, we identified three different sets of social studies departments: (a) those that placed 
special emphasis on higher order thinking, but organized instruction according to familiar 
patterns in the comprehensive high school (henceforth, the five "select" departments); (b) 

^Itoe rates of agreement are consstent with, but slightly lower than those achieved in 
NAEP scoring of persuasive writing. This was to be expected, because our scoring required 
more complicated judgements about stiutents' use of subject matter. 
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those that made no special department>wkie ^orts towaid higher order thinking and were 
also conventtonally organized (henceforth, the seven "representative" departments); (c) those 
that invoked a ctepartnMsntal emf^asis on higher order thinking and, in addition, had made 
significant changes in the organization of instructksn (henceforth the four "restructured" 
departments). Initial evidence of departmental emphasb on higher oider thinking was drawn 
from statemenu of the department chair, examination of course syllabi, and classroom 
observations and staff interviews compteted in a one-day, two-person site visit 

Because we sought an estimate of the highest tevels of classroom thoughtfidness, we 
concentrated on those tei^heis in each departmCTt who were identified by the department 
chair as most likely to emphasize higher order thinking. We also sought evidence that 
opportunities for thoughtfuhiess were available to all stuctenu, not only the high achiewrs. 
The department chair at each school setected three main courses, taught by different teachers, 
to be observed at least four times over the school year. The three classes were to illustrate 
as much higher order thinking as possible, but they were to include (a) a class with a 
substantial proportion of lower and mkldte achieving sttulents; (b) a histoiy course with a 
diverse range of students; and (c) any other class that best illustrated an emphasis on higher 
order thinking (the students in these were usually high achievers). Quantitative analyses were 
based upon four lesson c^servatiom. &om each of the classes m which students were tested. 

Due to limited funding for the project, it was not possible to administer pre and post 
tests of higher order thinking to students in all classes in the 16 schools. Instead, the 
assessment of student achievement was conducted in stages. There \ras no assessment of 
student achievement in the first phase investigation of the five select departments. During 
the second phase study of seven representative departments we used the posttest described 
in section III, but no comparable pretest. Thanks to collaboration with other researcheis in 
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Phase n, we liad access to two tests for ninth ^»leni «iministered in the faU that oiuldseive 
as controls for students' writing competei^ and social studies kix>wledge at eotiy. Tte first 
was a short answer test of social stiKlies ki«3wledge consttting of multiple-chokre and short- 
answer items drawn firom earlier N tests in social studies. For the second, students wrote 
an essay (in 15 minutes) about a place or a possession that was important to them and 
instructed to describe it "as fully as you can and explain wliy it is important to you." Since 
nei'.'^er of these required students to write persuasrvdy about constitutional ^ues> they can 
be consiftered "weak" pretests. In tl^ Phase m study of restructured departments we 
administered a far more rigorous pretest in the £alL Similar in form to the posttest, this 
pretest was a written eiiercise that asked stiKients to persuasively defend thdr position on a 
constitutbnal issue that invoked a school principal's censorship of an artick; in a student 
newspaper. The structure of ths exercise ym icU;ntk:al to the Karen Doctor case that was 
given as tte posttest 

To examine the relatbnship between classroom thoughtfulmss and students' 
persuasive writing on coi^titutiunal issues, we omsider three different data sets. Data Set 

1 consists of all students in Phas» n and in, without considering any pretest data. Data Set 

2 consists of the ninth graders in Phase n who took the "Sveak" pretests on social studies 
knowledge and writing. Data Set 3 cons^ts of aU students in Phase in who took the "strong" 
pretest on constitutional reaioning. 

V. ResBlts 

The central question is the extent to which classroom thoughtftiln^ (when measured 
by generic indicators) is associated with stwtents' higher order thinking (when measured by 
their persuasive writing about a constitutional issue). Before examining the findings, note that 
the design of the study work^ against the discoveiy of a strong relationship between the 
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dependent and independent variable. Pe^ormance on the posttest would seem to be 
maxunized by instruction related directly to the test, but noi^ of the teachers concentrated 
on the teaching of persuasive writing or on tte umterstanding of constitutional reasoning. 
Instead, instruction concentrated on topics typk»% pursi^ in the diverse courses observed, 
such as US History, World History, Politics, Sociok^, Economics and others. Furthermore, 
the independent variable of classroom thoughtfulness did not assess the quality of instruction 
for the specific competencies of persuasive writing or constitutional understaiuling. 

A posttest on student thinking related to teachen' specific content goals would have 
been preferred. This would allow one to test the more useful hypotheas that increased 
classroom thoughtfulness in the teaching of a topk^ as assessed by common indicators, will 
enhance student performaru^ in higl^r order thinking on that topic As e^cplained earlier, 
it was not possible to develop an a^essnM^nt exercue respoiuive to each te«:her's content 
goals. We were aware of the odds against finding a strong association between the generic 
qualities of thoughtfulness we observed and the more speciik competencies required for 
success on the posttest chosen for thb study, but we wcm hopeful nevertheless. 

The results are presented in Tables 2 • 4. The means and standard deviations of Data 
Set 1 (the full sample of students) in Table 2 indicate that performance on the higher order 
thinking task (Posttest Constitutional Reasoning) was barely minimaL Mcsl students (66%) 
were ghnen scores of 1 or 2; only 11% scores of 4, 1% scores of 5. Tnis confirms previous 
reports of low levels of student competence in writing about complex problems.^^ Levels 

Data Set 3, pretest and postest means were virtually identical Assuming that the 
tests were of equal difficulty, thu indkated ix> overall improvement in perfonnai«:e over the 
academic year. The assumption of equal diCBculty wk confirmed through a separate study 
in which the tests were randomly ass^ned at the same point in time to two grou{» of students 
(N=106), grades 9-12 from three high schools. While the means were somewhat higher than 
in Data Set 3, there was no difEerence in the means of the groups that took each test 
(posttest =2.43, pretest =144). 
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of Gass Tboughtfulness also tended towaid the lower end of the 5-pomt scale, with most 
students (72%) experiencing classes that scored below 3.5. Thb finding is consistent with 
other studies that have found low levels of cognitive work in high school classrooms. It is 
particulariy disturbing in this study in which teachers most likely to promote higher order 
thinking were deUberately sampled. 

Mean values for the stuctent background variables (sex, minority status, parent's 
educatk>n) were cbse to national norms. Students* grade point averages and the ability level 
of the classes clustered in the mid-range- The grade level of students (teacher-reported class 
average rather than student self>report) tended toward the kiwer levels, because of the large 
number of ninth graders in the Phase 11 data set. 

The correlations of most interest in Table 3 are those associated with Posttest 
Constitutional Reasoning and Class Thoughtf ulness. As expected, the posttest scores were 
most strongly related with the strong pretest, the pretest of social studies knowledge, the 
ability level of the class, and student grade point average. Students in the upper grades were 
more likely to do well on the posttest, and those in classes with higher percentages of African 
Americans were likely to do worse. The posttest correlated 37 with class thoughtfulness, but 
so did the pretest. These results suggest the possibility that instruction reflects students' 
initial achievement rather than influencing it. 

In other reports of this re&earch we deWe deeper into the pc»sible determinants of 
classroom thoughtfulness, by considering differences in thoughtfulness among teachers and 
schools and how these differences can be explained by characteristics of the teachers, the 
leadership and the organizations.^^ But correlations here indicate what might be expected; 



"See Ladwig (1991), King (1991), Newmann (1990b, in press a). Onosko (1989, 1990. 
1991). 
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namely, that thoughtfulness is higher in classes with older students (i.e., higher grade levels), 
in classes with larger percentages of higher achKving students, and in classes with students 
who perform better on the pretests. 

Possible explanations for the correlations with classroom thoughtfulness are that 
teachers' eicpectations for student performance influence the degree to which they promote 
higher order thinking, that teacher expectations are determined largely by their assumptions 
about student ability, and that these expectations are influenced by students' age and prior 
school achievement Expctctations based on these assumptions would result in younger and 
lower-achieving students' having fewer opportunities to experience thoughtful classrooms. 
It is encouraging, however, that levels of classroom thoughtfiUness were not highly associated 
with students* sex, parental education, or minority status. 

The regression results in Table 4 provide more information on ihei relationship 
between classroom thoughtfulness and student posttest scores. Each analysis offers a different 
way of examining the issue. Analysis 1 examines the relationship controlling for background 
variables, but not considering the influence of either type of pretest Here most of the 
background variables are associated with posttest performance, but the most powerful ones 
(considering standardized coefficients) arc gnwic point average, ability level of the class and 
grade level of the class. Holding background variables constant classroom thoughtfulness 
appears to make a difioence. The raw regression coefficknt indicate that an increase in one 
point on the thoughtfulness scale would, on average, be associated with a gain of a fifth of 



^^Uing attention to te«;her expecutions (an unnKasured variable) as a way of 
explaining these correlations is m)t meant to dssm^ or to underestimate the «:tual difrK:ulties 
te«:hers face in promoting higher order thinking with younger and kjw achieving students. 
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a point OR the posttest^^ Whether this is consideied "lar^* or "smaU" is a matter of 
interpretation. The total posttest variance e3q}Iained by tl^e variables is 34%. 

Analysis 2 includes two %veak" pretests completed by ninth graders and presents a 
somewhat different picture. Considering the standardized coefficients, we see that posttest 
performance is mc»t powerfully related to initial social studies knowledge and to grade point 
average, but that initial writing ability and classrooin thoughtfulness also have influence. In 
this analysis, perhaps due to cnntrok for pretests, the effects of class ability level, class racial 
compositktn, and parents' education are lower than in Model 1, and grade paint average also 
loses some importai»:e. In terms of raw coefficients, a 1 point differeiKc in classroom 
thoughtfulness is again jKssocial^d with a Gfth of a point on posttest. While total variance 
explained was less than 30%, the fact that classroom thoughtfulness survived the controls for 
social background and pretests is a potentially important result 

Analysis 3 offeis the nusst rigorous test of (bs associatk>n of classroom thoughtfulness 
with posttest, because of the inclusion of the strong pretest Controlling for the strong 
pretest, along with the other variabtes, eliminated the association of classroom thoughtfulness 
with pcsttest performance. Instead, the pretest, ability level of the class, grade level of the 
class, and student grade point average contribute, in roughly equal amounts, virtually all of 
the predictive pow^r, and this analysis explains more variaiv^ (3S^) than the first two. 

Considering the nature of the posttrat and the d^ign of the study, this finding might 
well be expected. On the other hand, sinc^ the first two analyses indicated a connection 
between classroom thoughtfulness and posttest performance, even after controlling for several 



^^o compare the magnitude of influence among in(k:pendent variables measured in 
different metrics, it is useful to refer to standardized coefficients. To estimate more 
concretely how a change in a given indepemSoit variable might affect a dependent variable, 
it is i»eful to refer to raw regression (^efficients. 
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background variables rarely included in analyses of instructional effects, w were reluctant to 
allow Analysis 3 to terminate the invesUgation. 

Could classroom tboughtfuloess affect performance on this exercise in ways other than 
those tested in the straightforward liiwar regresskms? Is it fXMsible, for example, that gradual 
increases in thoughtfulness bebw a certain threshold would make no difference, but that 
students exposed to the highest tevcb would perform better? We exptored iWs possibiUty by 
dMding the 20 classes that took the strong pretest (Data Set 3) into quintiles based on their 
class thoughtfiilness scores (which ranged from 2.72 to 433). We asked whether studente in 
the top fifth of the classes would perform belter than those ir. the tower four fifths. 
Regression analysis indicated no such result, which sustained the finding of analysis 3: in social 
studies classes focused neither on persuasive writing nor on constitutional reasoning, general 
quaUties of classroom thoughtfuhiess had no apparent impact on student pereuasive writing 
on constitutional ksues.^^ 

VI. COBCloSiOB 

This study attacked a perplexing problem. Social studies includes multiple fields of 
inquiry that have not been organized into a coherent, nationally accepted curriculum. In spite 
of lack of consensus on a core of essential content, there seems to be much agreemcm on 
at least three points: (a) Diversity ought to be preserved; (b) Regardless of what is studied, 
teachers ought to promote thinking rather than mindtess reproductfen of knowledge; and (c) 



. ^^^^ ^ Set 3 was limited by the number of classes. Although more 

than 300 students took pre- and post-tests, there were only 20 classes, thus only 20 distinct 
scores for classroom thoughtfiifaicss, and in the quinlile analysis only 4 classroom 
thoughtfulness scores per quintile. A better design for the study would have included many 
more classes (eg. 100) which took the strong pretest This wouki altow us to study (through 
hierarchical Ur^ modeling) the variation in association between classroom thoughtfiitows 
ami posttest that might be due to different types of classes while «multaneously analy^ng the 
effects of individual level variables (e.g. social background, gpa, and pretest) on pattest 
scores* 
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Students ought to be able to ckironstrate competence in analysis and interpretation of social 
phenomena. Recent national interest in accountabOity challenges social studies educators to 
develop common indicators of performance on the last two points: how well teachers teach 
thinking and how well students learn to think. 

This project proposed a conceptualizatk>n of higher order thinking and its promotion 
in the classroom that is applicable to a host of students, teachers, and topics of study. 
Similarly, it devek>ped an exercise to assess sti^nt competenoe in thinking about important 
social studies content - constitutional issues. The main empirical problem was, given the 
diversity in what the observed teachers actually Uught, whether the 'generic" qualities of 
classroom thoughtfuhiess that the project observed would be associated with student 
performance on the assessment task. 

Seventy-three classes in eleven high schook were obseived and ahsost 1400 hundred 
stuttents were tested. I>esign variatk^ns within the study led to the use of different dau sets 
to estunate the possible impact of cUissroom thoughtftiteess, but all analyses controlled for 
several background variables at both the individual and class level. Logistkal limitations 
resulted in only 20 of the classes and 340 students taking a pretest that made intellectual 
demands equivatent to the posttest 

Bo^use our measures of cl^room ttoughtfiihiess were not derived from teaching 
the specific competencies required on the posttest (le., undersUnding of constitutbnal 
reasoning and persuasive writing) and because mi teachers concentrates! instruction in this 
direction, we did not anticipate finding a strong relationship betM^n the main independent 
and dependent variables. We chose to investigate this relationship, nevertheless, for at least 
two reasons. 
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Fust, research on the importance of school and classroom culture suggests that 
student learning is influenced general qualities of human interaction, not simply by 
pedagogic moves tied to the te«:hing of specific content^^ The observation scheme used 
here offered an opportunity for quantitative exploration of the relationship between one 
aspect of classroom culture (thoughtfuliKss) and student performance that demanded complex 
thinking in social studies. 

Second, the recent concern with national education standards has highlighted political, 
professional, and technical issi»s related to the pluralistic nature of social studies instruction. 
This investigation could rontribute new information on the pn»pects of using a common 
c^»ervation scheme and a common assessment exerdse with classes that differ in the subject 
matter taught 

After controlling for students' sex, race, parents' education, gradepoint average, grade 
level in school, ability level and racial composition of the class, and pretests of social studies 
knowledge and writing ability, classroom thoughtfiihiess was associated with student posttest 
performance. When student performance on a pretest virtually identical to the postest was 
taken iiito account, classroom thoughtfulness had no association with posttest scores on 
persuasive writing about a constitutional issue. T»ie lack of relationship is consistent with the 
point we emphasized earlier: that success in meeting of higher order challenges in a specific 
content area demands in-depth knowledge in the area, not simply general skills and 
dispositions. The result seems consistent with research on instruction m a variety of subjects. 
Generic instructional traits are often not associated with complex intellectual performance in 



"See Lightfoot (1983), McNeil (1986). Mctz (1986), PoweU et al (1985), Rutter el al 
(1979), Scdlak et al (1986). 
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specific a)ntent cktmains. Instead, "what constitutes effective instruction...varies with context" 
(Brophy & Good, 1986, pJ70). 

What are the impUcatbns of these findings? There is a great risk that the findings 
will be misinterpreted as evidence that general qualities of classroom thoughtfulness do not 
enhance student achievement in social studies, and that, therefore, the dimensions of 
thoughtfulness we propose should not be pursued in teacluDg. This interpretation is 
misguid<xl in several ways. First, because the study examined only a veiy specialized form of 
social studies achievement (persuasive writing on constitutional issues), its findings should not 
be generalized to all forms of social studies ^hievement Second, since teachers did not 
teach the knowledge and skills needed for persuasive writing on constitutional issues, the 
study offered no evidence on the more appropriate question; namely, whether more and less 
thoughtful approaches to the teaching of persuasive writing on constitutional issues affects 
student achievement in this domain. Third, since the study did not assess student 
achievement in the specific domains that teachers d^ address, it offered no evidence of the 
impact of general dimensions of ttwughtfubiess on the quality of student achievement or on 
higl^r order thinking in the topics actually taught Given the study's silence on each of 
these important matters, it would be premature to use it as a basis for dismissing the 
importance of classroom thoughtfulness along the dimensions we proposed. 

On a more positive note, recall that in analyses 1 and 2, classroom thoughtfulness was 
associated with posttcst performaiu:e after taking into »xount several important background 
variables. Although each analysis failed to control for a rigorous pretest, the findings are 
(X)mpatible with the prospect that classroom tk^ughtfiilness would be more highly associated 
with student performance on a posttest of higher order thinking in the topics actually taught 
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To adeqiiately test the influence of classroom tboughtfuiness, a study that assesses student 
hi^r order thuiking in the topics taught is needed. 

We hoped that the project would contribute to practice, not simply by ofTering 
evkieiKS on empirical questions, but also by developing instruments to assess the 
tboughtfuiness of teaching and the performance of students on fades that require higher order 
thinking. Althou^ yitc identified generk qualities of classroom thoughthUness that might 
function as common indicators of teaching related to thinking, we did not develop a generic 
test few student thinking in social studies. The test of penuasive writing on constitutional 
issues calls for specialized competence; i.e. jurisprudential reasoning expressed in writing. 

In fact, to rely upon a single exercise to measure geserk thinking in social studies is 
probably misguided. Instead, it would seem more priKlent for future work on assessment to 
aim toward a diverse set of tasks. Ideally, these tasb would be sensith^ u leachers' diverse 
content objectives, but they could be scored according to generic qualities, such as depth of 
understanding or skill in sunmiarizing arguments. While success on the tasks would require 
domain-specific competence, a set of common scoring criteria would hope&iUy assess how 
students use in-depth knowledge, skills, and disposuons to solve the diverse kinds of higher 
order challenges that can enrich instruction in social studies. 
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Appendix 

REASONING ABOUT STUDENT LOCKER SEARCHES 

This reading presents a court case involving the search of a student's school locker by a school 
administrator. Though not an actual case, it is based upon cases presented to the U.S. Supreme 
Court You are to be the judge. As you read, be thinking about how you might (kcide this case. 
The qua irf STATE OF NEW Y ORK v.KAROf DOCTOR 

A teacher at a high school in New York discovered, Karen, a 16 year old sophomore and her 
friend smoking cigarette on school grounds in clear violation of a school rule. The teacher took 
them to the principaVs office. Karen denied that she had been smoking, .saying that she did not 
smoke at all The Assistant Principal, Mr. Hardy, then insisted on seeing the contents of her purse. 
He found a pack of cigarettes and also a package of rolling papers which are often used to smoke 
marijuana. He then decided to search Karen's kx:ker. 

With Karen present at the search, Mr. Harcfy discovered in her kxker a small amount of 
marijuana, a pipe, a note card with a list of students who owed her money, and two letters that 
indicated she was invoh^ in dealing marijuana. He then contacted the police and delinquency 
charges were brought against Karen. In court, Karen's lawyer argued that the search of her kxrker 
violate her constitutional rigjttts ami therefore Xh& evi(km:e found in her locker cannot be used. The 
case should be dismissed. The attorney representing the school and M**. Hanfy argued that the school 
had reasonable grounds for searching her purse and her kicker and therefore the evidence uncovered 
can be used in the trial She should be found guilty. 
Backsroand lofonaatloa 

The foUowing informatbn is provided to help you think about the case. Please read carefully. 
You should use this information in writing your argument 

AU ciUzem have certain rights which are guaranteed in the United States Constitution. The 
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Fbuith Amendment states. The right of ths people to be secure in their persons, papers, and effects 
against unreasonable searci^ and seizures shall not be Elated and no warrants issued, but upon 
prc^ble cause..." In other words, citizens have a right to priv«:y. Government officials and other 
authorities such as the fx?lu:e may not search any citizen or thdr personal possessions without good 
reason or "probable cause". For example, the police often must present evidence to a judge that 
something illegal is located in a specific place before they can conduct a search. The judge decides 
if the evidence b enough to justify the search, and if so, the judge Ksues a search warrant to the 
police. A search warrant is a document signed by a judge that gives authority to the police to search 
a specific place for specific items. 

As you consider the case, keep in mind that school officials are viewed by the Courts as a type 
of government official or authority. They have a responsibility to maintain order so that learning can 
take place, and a responsibility to protect students from harming themsehres or others. This may, at 
times, invoh« the search of students, their possessions, and their kx;kers. 

There are several general principles that the VS. Courts have used to help them decide cases 
involving the search of students and their possessions. These are called precc^nts. The following 
principles or precedents are summarized to help you decide the case of New York v. Karen Doctor. 

Fu5t, the Courts have developed the principle that school officials serve as representatives 
of parents during school-related activities, and, in some situations, have the right to act as a student's 
parent. 

Second, the Courts have decided that, unlike the police, school officials can conduct searches 
without a warrant if they have "reasonable suspicion" to believe something illegal or dangerous is 
present However, there must be evidence that something harmful is hidden by a student 

Third, tl^ Courts have decided that the danger of the items for which the search is conducted 
must be balanced against the student's right to privacy. Therefore, ::chool officials must decide how 
dangerous the item is before conducting a search. The student's age, hktoiy, and school rcxord, and 
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a tewrl^r's past experience with the student can piuvkte infonnatbn to decide if there is reasonable 
suspicion to conduct tfae search. 

Fourth, the Courts assume that student kxkers are different from a house, motor vehicle, 
backpack or even a rented private kx:ker. School kx^kers are to be viewed as having two owners, the 
student and the school. Lockers are owned by the school, but are assigned to students for their 
private use under the condition that dangerous or illegal items are not to be concealed. 

As judp?. you are to answer ths question: 

Dkl the school violate Karen^s constitutional rights by searching her purse and then her 

locker? 

Please write an argument to try to convince someone of your petition on this question. In 
your argument, you should 

-State your position on the question, 

--Support your position by giving as many reasons as you can, and 
-Explain why they are gooa reasons. 

Keep in mind that yova position will be most convincing if you include information from the 
reading and show weaknesses in the opposing positbn. Good luck! 
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Initia] Critnia for ClMsrooai Thoi«htfolBcss 

Classes were rated from 1-S. 1 ^ "very inaccurate* (tescriptioa of class; 5 ^ "Veiy accurate* 



^1. In this class, there was sustained examination of a few topics rather than a supe. udal coverage of 
many. 

*Z In this class, tte lesson displayed sutetantive coherence and oontinuiiy. 

*3. In this dass, students were given an appropriate amoimt of time to think, that is, to prepare responses 

to questions. 

4. In this class, the teacher carefully considered explanations and reasons for conclusions* 

In this class, the teacher asked chaltengi]^ questions and/or struaured challenging tasks (given the 
ability level and prepwation of the stwlents). 

6. In this class, the teacher pressed imUvidual stwtents to justify or to clarify their assertions in a Socratic 
manner. 

7, In this dass, the te»^ tried to get students to gen«atc original and unconventional ideas, 
explanations, or solutwns to problems. 

*& In thii dassroom, the teachn was a modd of thouf^tfiihiess, (Prindpal indications are: thete^er 
showed appredation for sti»lents' kleas and appreciation for alternative approaches or answers if 
tesed on sotmd reasoning; the te^^ explai»d tow be (she) thought f hrouf^ a pn^lem, tt^ teacher 
acknowledged the difiBculty of gsining a definitive understanding of the topic) 

9. In this class, students assumed the roles of questioner and critic 

^10. In this class, students offered explanatioiui and reasons for their conclusions. 

11. In this class, students ^nerated original and imoonventional ideas, c^lanations, hypotheses or 

solutions to probtems, 

IZ In this class, student contributions wit articulate, germane to the topic and connected to prior 

discussion* 

13. What proponfon of stiMlents were active partidpants? 

14. What proportion of time did students spend engaged in thoughtful discourse with each other? 

15. What proportion of sti^nts showed genuiw involvement in the topics discussed? (Cues include 
raising hands, attentiveness nmnifested by f^ial expression and body-langtia^, interruptions motivated 
by involvement, length of student respomea). 



^These variables are considered minimal requirements for a thoughtful iesson. 




JGknalllQn Vvrtebkm, Mlmm mnd Standsnl Drviatioiis for iuic^ Datii Sri 



Nol3«7 
K SD 



N-734 
X SD 



ilata set 3 
ST SD 



Cdc&RKuiacioil 
Male 



Afncao 
Amcncan 



Pwrem 



Or»)e 

Point 

Average 

CUtfi 

Uvrl 



nasi 
AbiUfy 



Percent 
Atnencaf) 



PosuTcst Pemiaifvc writing on 
Onuiitutkmal osue of sdKiol locker search, 
8a>red 1^5. 



SiiKiesit te%« measure by male 
femik « Ql 



1. 



2L21 



,48 



StiKknt race, self -report, measured by .14 
African AnKrican « 1, other 0. In this 
study* most mm-white stiKknts were African* 
American, 

Parcnu* education measured on a five point .1.03 

Bcate (1 n less than high schoc^ graduatkm. 

2 » high schod graduation cmiy, ... 

S » graduate or pnrfciskmal degree) and 

averaged between two parents, student report. 

Stu(fem*s gra<te point average measured by 5.63 
the siudent*s self repc»t (m an eight-pmnt 
scale (1 = mostly below D to 8 » mostly A). 

Mean grade level of stuctenu in the class« 9.72 
based on teacher report of percent of 
students at each level, grades %IZ 

Mean ability of students in the elm, based 105 
on teacher report of percent cl students in 
the knmx middle (2) and hsghot (3) 
thirds of school achievement, 1*3. 

Percentage of African Amerkan students 17.73 
in the class mxxmiing to teachcn' reports. 



1.01 



.48 



.33 



1.10 



1.58 



.53 



20.17 



i04 .91 

.47 .50 

.12 .32 

3,32 1.04 



5.57 



1.99 



13.66 



IJ3 



,50 



13.2!» 



2.22 sn 



.53 ,45 



.20 .36 



Z09 .83 



5.80 



10,33 



2.vn 



30,73 



1.51 



.93 



39 



28.81 



rreiess Pre-Tcst. SfKjal Studies Kncnvicdgc. scured 

Knowledge 0-79. 

Pretest Prc-Tcst. Wming Ability, scored 0-9. 
Wnung 

Pietc« Pre-Tc»t Penuasive writing on 

ConsiriuunQBi Constitutkmal Issues of Schtx)} Cemorsbip, 

Rnsom&i scored 1-5, 

Class Ttxwi&s- aamoom TboughtfuJiKss* n^an of six 

tufatu indicaton scored 1-5. 



3.12 



.66 



57,46 



5,50 



Z83 



1184 
1..35 



56 



Z27 



3.47 



.93 



.51 



Data set 1 Ail studenu in the study who look the post^test. 

Data set 2 Ninth grade students in 7 "represcDtative' 
scIhxjIs. 

Data set 3 All students in 4 "restructurt^" schools. 



^ Alt students were in the ninth grade. 
^ Test not gh«n to this sample of siiKients. 



Table 3. 

CofrrfaUons Among all Var&d)ks (Data set I, N « 1387)* 



Malt 



An 




PoHtcs OmMAuiiansI l.UO 



-.13 
.20 
.2«) 
41 
.29 
.4.1 



Afncan AnKncan 

Cirade hunt Awrrajtc 
f1a» level 
ClBM Atotity 
PcrcrM African American -.W 

PrrtcM Wnimg 29 



|*re(c«i CiVBlitutKinal 



.4? 



1.00 
.(12 

-.m 

.10 
.01 
-.09 

.09 
IK 

.10 



1.(10 
-.12 
-.14 

m 

-.17 
-.30 
.21 



1.00 
.20 
.11 
.15 

-.25 
.25 
.19 
.25 



1.00 
.09 
M 

-.19 
.44 
.26 
.28 



l.(N1 

.18 
.00 



.22 



l.UO 

.41 

.24 

m 



l.flO 
-35 

-.11 

-.30 



1.00 
.29 



i.on 



1^ 



.17 



-.07 



.10 



.24 



49 



..18 



-.09 



.11 



.14 



.17 



1.00 



*C«rrelati<ins Pretoit Kmmicdf^ and Pretext Writinf; based on data set 2. N « 7.14; ali studcnu gra«k; 9. Correlalkiitt fnr Pretest Comtitulionai Re«»ning based on data set 1, N = 342; 
Prcl»»l KmiwknJfic and Pretest Writing nnl administered to data jsct 1. 



TBble4 

Regression of PiHttest of ConstitBtloBa] Reasoning on Class Tlionghtflilness, 

Background Vulablcs and Pretests 



Analysis 
Model 1 
(Data set 1) 
Beta^ 



Analysis 
Model 2 
(Data set 2) 
B Beta F 



Analysis 
Model 3 
(Data set 3) 
B Beta P 



Male 



A6icao American 



Parents Educatkm 



Grade Point Average 



Class Level 



Class Ability 



-.14 
(.05^ 

-.18 
(.07) 

.09 

(.02) 

.15 
(.02) 

.19 
(.03) 

.35 

(.05) 



.06 .00 



Percent Afhcan-American -.00 

(.00) 

Pretest Knowledge 



Pretest Writing 

Pretest Constitutional 
Reasoning 

Qass Thoughthilness 



(Constant) 



Variance Explained 
(adjtisted R^) 



.20 
(.04) 

-1.91 
(.26) 



-.06 
.10 
.23 
.18 
.18 



.01 
.00 
.00 
.00 
.00 



-.11 .00 



.13 .00 



.00 



-.15 
(.06) 

-.14 
(.09) 

.04 
(.03) 

.09 
(.02) 



.11 
(.07) 

-.00 
(.00) 

.02 
(.00) 

.08 
(.02) 



.21 
(.06) 

-.63 
(.25) 



-.08 .01 



-.05 .14 



.05 
.16 



.18 



.00 



.06 .11 



-.05 .19 



.22 .00 



.12 .00 



.13 .00 



.01 



-.10 
(.09) 

.10 
(.13) 

-.03 
(.05) 

.12 
(.03) 

.20 
(.05) 

.37 
(.09) 

-.00 
(.00) 



.21 
(.05) 

.10 
(.10) 

-1.90 
(.61) 



-.05 



.04 



.19 



.23 



.05 



.28 
.43 



-.03 .57 



.19 .00 



.00 
.00 



.13 .02 



.21 .00 



.28 
.00 



.34 



.28 



.39 



^ B = raw regression coefficient 
2 Beta = standardized regression coefficient. 
^ P = probability due to chance. 



^ ( ) = standard error 
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